Accurate Capture of 3D Full-body Motion Using A Single Camera

In the past decade, motion capture technologies have enabled tremendous advancement in creating realistic human characters for virtual worlds, performing biomechanical studies of human movement, and providing natural user interfaces for interacting with computers, robots, and machines. However, current motion capture technologies are often limited to high-end applications because they are restrictive, expensive, and require special skills to set up and operate. This dissertation explores a new generation of motion capture technologies that address such challenges. We focus our study on two important and challenging motion capture problems: high-fidelity motion capture using a single video camera and online motion capture using a single depth camera. We first introduce a new video-based motion capture technique for reconstructing physically realistic full-body motion from single-camera video streams such as Internet videos. During reconstruction, we leverage Newtonian physics, contact constraints, and 2D image measurements to simultaneously reconstruct full-body poses, joint torques, and contact forces. across an entire sequence. For online applications, we develop a motion capture system that accurately captures 3D full-body movements in real time using a single depth camera. Both systems are appealing for home use because they are low-cost, easy to set up, and allow for accurate motion capture even with significant occlusions. For both systems, we assess the quality of the reconstruction results by comparing against those created by a commercial optical motion capture system. We demonstrate the quality of the reconstructed motions created by our systems is comparable to commercial motion capture systems, but our systems are far less expensive, restrictive, and cumbersome. More information about this dissertation can be found in digital repository at Texas A&M University: http://hdl.handle.net/1969.1/ETD-TAMU-2011-12-10566

How to get research into practice : first get practice into research

sch_iih2007 Jpub2738pub

Efficient Modeling of Future Context for Image Captioning

Existing approaches to image captioning usually generate the sentence word-by-word from left to right, with the constraint of conditioned on local context including the given image and history generated words. There have been many studies target to make use of global information during decoding, e.g., iterative refinement. However, it is still under-explored how to effectively and efficiently incorporate the future context. To respond to this issue, inspired by that Non-Autoregressive Image Captioning (NAIC) can leverage two-side relation with modified mask operation, we aim to graft this advance to the conventional Autoregressive Image Captioning (AIC) model while maintaining the inference efficiency without extra time cost. Specifically, AIC and NAIC models are first trained combined with shared visual encoders, forcing the visual encoder to contain sufficient and valid future context; then the AIC model is encouraged to capture the causal dynamics of cross-layer interchanging from NAIC model on its unconfident words, which follows a teacher-student paradigm and optimized with the distribution calibration training objective. Empirical evidences demonstrate that our proposed approach clearly surpass the state-of-the-art baselines in both automatic metrics and human evaluations on the MS COCO benchmark. The source code is available at: https://github.com/feizc/Future-Caption.Comment: ACM Multimedia 202

Research on an alternative method of turbine motor signal

In the modern mortar radio fuze, the use of turbine generators as the power source for fuzes is very common. The ballistic air pressure during the flight of the projectiles is used as the driving force to drive the turbine motor. In this paper, the turbine motor signal is parameterized in combination with the actual situation, and the idea of using the hardware to simulate the turbine power generation is proposed. The generation of the turbine motor signal is simulated by means of simulation software. Design the circuit to verify the simulation results, and have a certain reference for how to easily detect the fuze in the mass production process